The present invention relates generally to image processing, and more particularly relates to a system and methods for mixing two independent signal sources while smoothly controlling the rate of change during mixing, especially useful in scene-by-scene color correction systems, matte mixing, and xe2x80x9cblue screenxe2x80x9d video masking applications.
Video signal color correction systems for creating, enhancing, compressing, filtering, or otherwise modifying characteristics of video images are known in the art. Systems for modifying video signals are used for creating special effects as well as for correcting video images to compensate for variations in cameras, film, lighting conditions, etc.
Video image processing systems are especially employed in post-production color correction systems for motion picture film and/or video tape color corrections, and typically operate on a scene-by-scene basis. A xe2x80x9cscenexe2x80x9d is a sequential collection of images, often shot from the same camera, having the same viewpoint, composed in a certain way, etc. An operator using a typical post-production color correction system observes a target frame of a scene on a video monitor, adjusts the color or other parameters of the frame until it is aesthetically satisfactory, and stores color correction parameters in system memory. The color correction system preferably automates the application of the stored color correction parameters to the other frames of the scene.
In many applications, color correction is applied to selected regions of a video image. Various methods are employed to isolate color regions or geometric regions for receiving color corrections. A color correction system that isolates regions by color or hue is described in U.S. Pat. No. 6,337,692 entitled xe2x80x9cPrimary Color Manipulation Using Hue, Saturation, Luminance and Area Isolation.xe2x80x9d The system described in this patent isolates one or more color correction regions through operations performed in the hue domain. The regions for receiving color correction or modifications are defined through xe2x80x9cqualificationxe2x80x9d. Specifically, parameters of hue qualification, saturation qualification, luminance qualification, and an optional alpha filter are combined to define an alpha qualification function that isolates a region or xe2x80x9chue sectorxe2x80x9d for color correction. The alpha qualification function may have a shape in the hue domain that ramps or softens the applied color correction by generating a gradual or fractional transition towards the edges of the corrected hue sector.
It is also known in the art to define geometric regions for receiving color corrections. For example, U.S. Pat. No. 6,097,853 entitled xe2x80x9cUser Definable Windows for Selecting Image Processing Regions,xe2x80x9d describes a system where a user of an image processing system such as a scene by scene color corrector can define a window or region for purposes of applying image processing only to selected regions of an image. A geometric region can consist of a square, rectangle, triangle, or a composite geometric region defined using tools.
Color corrections can be effected for regions inside a defined window or regions outside the defined window. Furthermore, separate sets of color corrections can be defined for regions inside the window and outside the window. Utilization of windows for defining regions for receiving color correction is also called xe2x80x9cmaskingxe2x80x9d, and the window is often called a xe2x80x9cmaskxe2x80x9d or xe2x80x9cmattexe2x80x9d. Masking can be effected both with color region isolation or with geometric windows.
A matte or mask can be set up so that the regions inside the matte receive one set of color corrections, while regions outside the matte receive a different and second set of color corrections. As in the case of detecting a hue region for color correction with qualification, the boundaries of a window (or mask or matte) are often softened electronically with a gradual fractional application of color correction so that the transition between regions that receive different color corrections is not as noticeable. Stated in other words, color corrections in a transition region are gradually diminished or increased, as appropriate, between the inside and the outside of the matte.
The basic color masking technique is often employed in xe2x80x9cblue screenxe2x80x9d applications to create special effects. A xe2x80x9cblue screenxe2x80x9d application is when a camera shoots a scene with an actor positioned before a blue screen, with the system creating a xe2x80x9cmaskxe2x80x9d defined by the blue color. It is known in the video and film arts to shoot an actor in front of a having a predetermined blue color to derive a first video signal, provide a second video signal containing a background image against which the actor is to be superimposed, and to mix the first video signal with the blue screen with the second video signal. A color correction system detects regions of the first video signal containing the predetermined blue hue and adds or mixes in the second video signal in such detected regions. The effect is an overlay of the actor over the scene represented in the second video signal. In this manner, the actor can be made to appear superimposed over a different and independent image, e.g. a weather map, a scene from outer space, flying over the ocean, etc.
The blue screen technique is also known as xe2x80x9ckeyingxe2x80x9d, in the sense that the first video signal with the blue region serves as a key, and the second video signal provides a signal source that is substituted for the predetermined values of blue as detected in the circuit. Another term used by those skilled in the art is xe2x80x9cmattexe2x80x9d.
When conducting color correction operations, including keying, matting, and blue screen methods, it is generally necessary to mix two independent video signals to arrive at a combined video signal. In some applications, the two mixed video signals each contain a key or matte. A video signal mixer is usually employed to effect this mixing function.
Refer now to FIG. 1 for an example of mixing a first video signal containing a first key or matte and a second video signal containing a second key or matte. A first video image 12 containing a geometric matte or key 14 with a fractional boundary or transition zone 16 is to mixed with a second video image 20 containing a second matte or key 22 with a second fractional boundary or transition zone 24. The desired result is a third video image 30 where the two mattes 14, 22 are combined.
Assume further that the peripheral edges 16, 24 of the keys 14, 20 are fractionalized, i.e. the boundary between regions of the video image inside the key and outside the key experiences a tapering off of the application of color correction. Color correction applied to regions inside the key gradually tapers to zero in the fractionalized areas. Conversely, color correction applied to regions outside the key (if any) gradually tapers to zero in the fractionalized area.
The video images in FIG. 1 are xe2x80x9cwindowingxe2x80x9d signals in the context of video signal color correction signals. A windowing signal is a gray scale signal in that it goes from white (0) to black (1) with a number of fractional (gray) values in between as it transitions. The fractional values occur in the peripheral edges 16, 24 of the keys 14, 20. A window signal of white (0) would produce a color correction pertaining to an OUT-WINDOW set of color corrections as defined by the user, and a window signal of black (1) would produce an IN-WINDOW set of color corrections at the output. Any fractional window value (between 0 and 1) would produce a mixture of the IN-WINDOW and OUT-WINDOW sets of color corrections.
When mixing mattes that include fractional transition zones or regions, artifacts can occur where there are discontinuities between the fractional values defining the transitions. These discontinuities are most likely to occur in applications where two or more mattes are combined to form a larger composite area, such as shown in FIG. 1 at 32, which is the intersection between the window or matte 14 and the window or matte 22. Relevant portions of the signals that form the combined area 32 are shown in FIG. 1 as input signal F1 and input signal F2.
Various types of video signal mixers are known in the art, but suffer from certain drawbacks. An additive (or subtractive) mixer produces its output by adding or subtracting, as appropriate, the input signals and limiting the output. An additive mixer known in the art is shown in FIG. 2.
If the input signals F1, F2 from the images 12 and 20 respectively in FIG. 1 containing mattes 14 and 22 are mixed with an additive mixer such as shown in FIG. 2, an output such as shown in FIG. 3 is produced. The output shown in FIG. 3 includes noticeable crease lines 35 which result from discontinuities from adding the values in the fractional areas. These discontinuities are undesirable and are noticeable in certain applications.
Another type of video signal mixer that is known in the art is a non-additive mixer or NAM. This type of mixer, as shown in FIG. 4, produces an output signal by comparing the input signals F1 and F2, and producing an output comprising the greater (or lesser) of the two input signals, as appropriate. The output produced by a non-additive mixer for the example of FIG. 1 mixing F1 and F2 is shown in FIG. 5. Note the crease line 35 that results with the use of this type mixer.
FIG. 6 illustrates a multiplicative mixer which produces its output by multiplying the input signals F1, F2. The output of the multiplicative mixer for the example of FIG. 1 and signals F1 and F2 is shown in FIG. 7. A multiplicative mixer also produces crease lines 35 which result from discontinuities in the signal values.
In all of the prior art mixers described above, a visual artifact in the form of a crease line is created at the corners where there is a jump or discontinuity in the rate at which the output changes, resulting from the mathematics of the mixing operation. Color correction changes therefore will change abruptly at such crease lines, and these abrupt changes are often noticeable in the mixed output. Therefore, there is a need for a video signal mixer which does not produce such artifacts and results in smooth mixing of two independent video signal sources.
The present invention overcomes the disadvantages described above in video signal mixing circuits. Briefly described, the present invention provides a signal mixer for mixing a first video signal with a second video signal while smoothly controlling the rate of change of an output signal. An input stage receives the first video signal and the second video signal. The mixing circuit mixes the first video signal with the second video signal based on a predetermined parabolic function. An output stage provides a xe2x80x9cparabolizedxe2x80x9d output signal, i.e., the output signal is a mixture of the input signals with a parabola fitted between the two signals in transition areas. As a result, the output signal, which comprises the mixture of the first video signal and the second video signal, eliminates discontinuities in regions of the signal which would otherwise produce discontinuities in prior art types of video signal mixers.
More particularly described, the video signal mixer comprises a maximum signal detector for detecting the maximum value of the first video signal or the second video signal, a minimum signal detector for detecting the minimum value of the first video signal or the second video signal, and a circuit for providing a difference signal comprising the difference between the maximum value of the first video signal or the second video signal and the minimum value of the first video signal or the second video signal. An aperture circuit responsive to an aperture signal and to the difference signal provides an aperture varied signal for varying the output signal corresponding to the value of the aperture signal. This effects a degree of operator control over the extent of the parabolization.
The video signal mixer further includes a multiplier circuit for squaring the aperture varied signal, and an adder circuit for adding the squared aperture varied signal to the maximum value of the first video signal or the second video signal. The preferred output stage of the video signal mixer comprises an output multiplexer for selecting either the maximum value of the first video signal or the second video signal or the output of the adder circuit to provide the parabolized output signal.
The present invention further provides a method for mixing a first video signal with a second video signal to provide a parabolized output signal while smoothly controlling the rate of change of the output signal. The disclosed method comprises the steps of detecting the maximum value of the first video signal or the second video signal, and detecting the minimum value of the first video signal or the second video signal. The difference between the maximum value of the first video signal or the second video signal and the minimum value of the first video signal or the second video signal is then determined and provided as a first difference signal. A operator controlled aperture signal is provided for varying the mixer output and effecting a degree of control over the parabola that is fitted between the two input signals. The difference between the aperture signal and the first difference signal is calculated to provide an aperture varied signal. The aperture varied signal is then squared and added to the maximum value of either the first video signal or the second video signal. Finally, either the maximum value of the first video signal or the second video signal or the output of the adding step is selected to provide the parabolized output signal.
With such a construction and methodology, two independent video signal sources may be mixed with a smoothly varied output comprising a mixture of the two signals. The invention is especially advantageous for use when it is necessary to combine video images containing different keys or qualifiers, e.g. when the two separate video images to be mixed have intersecting mattes or keys.
These and other features and advantages of the invention may be more clearly understood and appreciated from a review of the following detailed description of the disclosed embodiment, and by reference to the appended drawings and claims.